Evolving Artificial Neural Networks Using Simulated Annealing-based Hybrid Genetic Algorithms

Artificial neural networks are among the most effective learning methods currently known for certain types of problems. But BP training algorithm is based on the error gradient descent mechanism that the weight inevitably fall into the local minimum points. It is well known that simulated annealing (SA) and genetic algorithm (GA) are two global methods and can then be used to determine the optimal solution of NP-hard problem. In this paper, due to difficulty of obtaining the optimal solution in medium and large-scaled problems, a hybrid genetic algorithm (HGA) was also developed. The proposed HGA incorporates simulated annealing into a basic genetic algorithm that enables the algorithm to perform genetic search over the subspace of local optima. The two proposed solution methods were compared on Rosenbrock and Shaffer function global optimal problems, and computational results suggest that the HGA algorithm have good ability of solving the problem and the performance of HGA is very promising because it is able to find an optimal or near-optimal solution for the test problems. To evaluate the performance of the hybrid genetic algorithm-based neural network, BP neural network is also involved for a comparison purpose. The results compared with genetic algorithm-based indicated that this method was successful in evolving ANNs.